Since the invention of the stereoscope in 1847, there has been a desire for emulating the 3-D images of nature instead of being content with two dimensional images which lack realism due to the absence of depth cues. Many techniques have been devised and developed for producing 3-D images, each varying in degree of success and quality of image. These techniques generally belong to two major classes, namely the autostereoscopic imaging class which produces 3-D images which can be viewed freely without spectacles, and the binocular stereoscopic imaging class which produces 3-D images which requires observers to wear spectacles or viewers. Techniques of the later class have been found in 3-D movies of the 1950's and in occasional 3-D image productions such as 3-D comic books.
Color separation of stereo images has been utilized for over fifty years in the production of photographs, 3-D movies and the printed page. In prior art devices such as shown in U.S. Pat. No. 3,712,119, stereo images are separated by mutually extinguishing filters such as a blue-green lens filter over one eye and a red filter over the other eye. With this combination, a full true color image is not obtained, and this color combination may cause eye fatigue, and color suppression.
In the prior art an object of a single pure color matching the filter color e.g. red or blue-green, would be transmitted only to one eye and so would not appear in stereo. However, pure colors are rare, and most objects are off-white, or pastel shades and so contain all three primary colors. Thus, most objects will have some component of each color and this enables the separation of right and left stereo images.
Prints, drawings or representation that yield a 3-D image when viewed through appropriately colored lenses are called anaglyphs.
An anaglyph is a picture generally consisting of two distinctly colored, and preferably, complementary colored, prints or drawings. The complementary colors conventionally chosen for commercial printings of comic books and the like are orange and blue-green. Each of the complementary colored prints contains all elements of the picture. For example, if the picture consists of a car on a highway, then the anaglyph will be imprinted with an orange car and highway, and with a blue-green car and highway. For reasons explained below, some or all of the orange colored elements of the picture are horizontally shifted in varying amounts in the printing process relative to their corresponding blue-green elements.
An anaglyph is viewed through glasses or viewers having lenses tinted about the same colors used to prepare the anaglyph (hereinafter, "3-D glasses"). While orange and blue-green lenses are optimally used with an orange and blue-green anaglyph, red and blue lenses work satisfactorily in practice and apparently are conventionally used.
The orange elements in the picture are only seen through the blue lens, the red lens "washing out" the orange elements. For the same reason, the green-blue elements are only seen through the red lens. Hence, each eye sees only one of the two colored pictures. But because the different colored elements are horizontally shifted in varying amounts, the viewer's eyes must turn inward to properly view some elements, and turn outward to properly view others. Those elements for which the eyes turn inward, which is what the viewer does to observe a close object, are naturally perceived as close to the viewer. Elements for which the viewer's eyes turn outward are correspondingly perceived as distant. Specifically, if the blue lens covers the viewer's right eye, as is generally conventional, then any blue-green element shifted to the left of its corresponding orange element appears to the viewer as close. The element appears closer the greater the leftward shift. Conversely, as a green-blue element is shifted only slightly leftward, not at all, or even to the right of its corresponding red element, that element will appear increasingly more distant from the viewer.
In addition to horizontally shifting the element pairs relative to each other, some users of anaglyphy for comic books also vertically shift the element pairs a slight amount relative to each other. Those users believe that the slight vertical shift improves the 3-D effect.
Normally 3-D images appear monochromatic when viewed through 3-D glasses.
Three dimensional techniques are closely related to the psychology and physiology of an observer's cognitive processes. Subtle changes in selection of portions of the spectrum presented to each eye can result in significant changes in the observer's perception. Even when viewing the same 3-dimensional image through the same viewers, different observers may perceive a 3-dimensional image in different ways.
Many viewers have difficulty when viewing 3-dimensional images and cannot view the images for any significant length of time without unpleasant side effects, such as headaches.
The transmission of programming, in particular, video programming where wide bandwidth is required, has been known to include a variety of compression techniques. Modern decoders of compressed video information will adapt their operating modes in dependence upon the properties of the incoming signal. It is also often the case that video programming is distributed in multiplexed format which requires that either the decoder or an associated demultiplexer be aware of where in the multiplexed stream particular programs are located.
One of the widely accepted standards for the distribution of video programming is that promulgated by the Organisation Internationale de Normalisation (International Organization for Standardization) under Standard ISO/IEC JTC1/SC29/WG11 entitled "Coding of Moving Pictures and Associated Audio". The particular working group responsible for that standardization effort is also known as the Motion Picture Experts Group.
MPEG compression and decompression standards have been evolving. The current advanced version is generally known as MPEG-2. The MPEG-2 standard is hereby incorporated by reference in its entirety. In the language of the MPEG-2 standard, a "program" is comprised of typically a source of video and a source of audio which share a common time base. One or more elementary digital streams (e.g. video or audio), as well as other data, are combined into single or multiple streams which are suitable for storage or transmission. System coding follows the syntactical and semantic rules imposed by the MPEG-2 specification and provides information to enable synchronized decoding without either overflow or underflow of decoder buffers over a wide range of retrieval or receipt conditions.
Under the MPEG-2 standard, an incoming individual video signal and related audio signal are encoded and packetized into a respective Video Packetized Elementary Stream (PES) and Audio Packetized Elementary Stream. The video and audio PES from one or more programs are combined into a transport stream for transmission or storage.
The transport stream is designed for use in environments where errors are likely, such as storage or transmission in lossy or noisy media. Transport stream packets are 188 bytes in length. Transport stream packets are generally divided into two sections. The first section constitutes header information and the second constitutes payload. Header information includes, inter alia, a synchronization byte, transport scrambling control and a thirteen bit program identification (PID) indication. PID value 0 is reserved as an indication that the packet includes program association table data. PID value 1 is reserved for conditional access such as encryption. PID value 0.times.1FFF is reserved for administrative (no-op) packets utilized for synchronizing the link. Other program identification numbers are utilized to identify transport packets with the program source from which they originate.
FIG. 1 depicts a transport stream utilizing the MPEG-2 standard. As indicated above, each transport packet in the transport stream carries a program identification number (PID). PID 0 carries program association table information. As illustrated in FIG. 1, a program association table maps a particular program source with the location of the PID associated with a program map related to that source. Thus, the program association table defines the location of program maps for each source of programming in the transport stream. The program map for CBS is found in PID 132; the program map for NBC is found in PID 87 and so forth.
The program map for CBS illustrated in FIG. 1 as that associated with PID 132 identifies the PID numbers for the packetized elementary streams (PES) for the video and audio channels associated with the program. One should note at this time that more than two PID's may be associated with a program. For example, there could be a data channel associated with the program which would include data for closed captioning for the hearing impaired. There could be a number of audio elementary streams for, for example, respective different languages. Thus the programming map provides some flexibility beyond that required for merely associating a single video and audio elementary streams.
Once the program map is known, the program decoder can be set to extract the desired elementary streams (e.g PID 56 in FIG. 1 and PID 74 shown in FIG. 1) for decoding and presentation. Transport packets of PID=56 represents the video elementary stream associated with CBS' MPEG video system. Within CBS' video elementary stream are video sequence headers which define things like frame rate, resolution, and the coordinates on the screen where display of the image should begin. Such coordinates are useful, for example, in defining pictures within a picture when multiple pictures are superimposed. After the video header sequence, the normal video syntax occurs which, in the case of MPEG, includes the normal frames associated with video compression such as I frames and B frames used in MPEG.
The elementary stream with PID=74 is the audio stream associated with the video shown in transport packets with PID=56 as defined in program map at PID=132.
A typical MPEG-2 transport demultiplexer and decoder is shown in FIG. 2. The data link specific interface 210 converts the incoming data link format (e.g. optical fiber encoded data) to a form (e.g. electrical) for demultiplexing and decoding. Transport stream demultiplexer 220 monitors the PID's of the incoming transport stream packets to identify those selected for decoding and routes those to respective video (230) and audio (240) decoders. Since the video and audio associated with the same program are derived from a common clock, clock control 250 drives both decoders. Note that the transport stream demultiplexer includes functionality, such as that described in connection with the description of the transport stream of FIG. 1, necessary to identify the particular PID's needed to extract information from the transport stream to present the desired program to the user.
Referring again to FIG. 1, when a subscriber tunes, for example, to CBS for the first time, before decoding can take place, a program association table and the program map for the selected channel must be received in the stated order from the transport stream. Thereafter, a packet from the video elementary stream must arrive with header information about the frame rate, resolution and coordinates for display of the image and the like before the decoder can be set up to actually decode the payload data associated with the video elementary stream.
Returning to the three dimensional imaging, one problem common to most observers arises when viewing a pure red or pure blue region of a 3-dimensional image through red/blue 3-dimensional glasses. In such circumstances, one eye will perceive black and the other eye will perceive nothing. This has a psychological and/or physiological impact on the viewer which most viewers find disturbing.
Further, when observing 3-dimensional images in which the left and right images are captured using complementary filters, the images reproduced in the colors of the filters, and viewed through viewers of the same colors (e.g. red/blue glasses) which separate the images, 3-dimensional images appear only in black and white. That is, color information is lost in the preparation of the 3-dimensional image. This is characteristic of most 3-dimensional images.
When processing color images using computers, it is common to separate an image into (e.g.) red, green and blue image components. Commonly each color component is referred to as an image plane. In the display of color images on a color cathode ray tube it is common to apply information from each color image plane to a respective electron gun of the cathode ray tube.
Normally, in the past, when preparing 3-dimensional motion pictures, the anaglyph frames were prepared in the post production suite.
When color images are captured, it sometimes occurs that one of the colors utilized for representing the image may be overexposed or underexposed as reflected, inter alia, in an inadequate dynamic range for that color. That is, anytime the color appears at all, it appears at maximum value or anytime it appears it appears at some minimum value instead of being spread over the entire dynamic range of representation. This adversely affects the quality of 3-D image produced.
In certain instances, for example in some video games, perspective views are generated from 3-dimensional wireframes. It would be desirable to be able to selectively implement either 3-dimensional perspective views of 3-dimensional stereo views.
The prior art generally required complex specialized equipment for the transmission of 3-dimensional images. This inhibited the use of 3-D technology because much capital investment has been devoted to equipment for handling regular 2-dimensional images. It would be desirable to utilize 2-dimensional transmission equipment to produce 3-dimensional images. It would also be desirable to have simple transmission equipment and reception equipment which would be suitable for distributing 3-dimensional images.